Stable Recovery of Shape and Motion from Partially Tracked Feature Points with Fast Nonlinear Optimization

The linearized approach to the shape from motion problem, e.g. the factorization method, is robust to find a unique solution with fast computation, but the occlusion and perspective distortion are out of scope in the linear formulation. In contrast, the nonlinear approach is free from such limitations, yet it involves two problems; one is to find the globally optimal solution and the other to reduce the computation time. In this paper, we present an effective nonlinear optimization method to recover 3D shape and motion. To overcome the shortcomings of the nonlinear scheme, we propose “double search (DS) procedure” and “preconditioned conjugate gradient (PCG) algorithm”. The DS procedure enables us to find two major solutions that correspond to the true and false shapes and then we select the globally optimal solution by evaluating the error of them. The PCG algorithm is an improved CG one whose computational performance is several times faster than the conventional LevenbergMarquardt (LM) algorithm. We carried out experiments with simulation and real data, and the results have demonstrated that the proposed method allows us to obtain the correct shape and motion with 3-9 times faster computation than the LM algorithm. Finally we have shown the applicability of the method to a large building reconstruction from a set of partially tracked feature points.